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PRELIMINARY AMENDMENT BEFORE EXAMINATION 

Honorable Assistant Commissioner for Patents 
Washington, D.C. 20231 

Sir. 

Before examination of the continuation application filed concurrently 
herewith under 37 CFR 1 .53(b), kindly amend the application in accordance with 
the following particulars: 



IN THE SPECIFICATION (35 U.S.C §120 Priority Statement): 

Page 1 , line 3, after the title and before the heading "BACKGROUND OF THE 

INVENTION", insert the following sentence: 

--This application is a continuation of U.S. Patent Application Ser. No. 

08/917,341, filed August 26, 1997, pending.-. 



IN THE CLAIMS : 

Please cancel claims 2-4, 7-17, 20-22, 24-30, and 32-34, without prejudice 
or disclaimer. 



Continuation of S.N. 08/917,341 



Please amend claims 1,18, and 31, as follows: 
-1 . (Amended) Apparatus for carrying out communications over a multi-tier virtual 
private network, said network including a server and a plurality of client computers, 
the server and client computers each including means for transmitting data to and 
receiving data from an open network, comprising: 

means for intercepting function calls and requests for service sent by an 
applications program on one of said client computers to a lower level set of 
communications drivers; [and] 

means for causing an applications level authentication and encryption 
program in said one of said client computers to communicate with the server, 
generate [said] a session key, and use the session kev generated bv the 
applicatio ns level authentication and encryption program to encrypt files sent by 
the applications program before transmittal over said open network. 

1 8. {Amended} Computer software for installation on a client computer of a multi- 
tier virtual private network, said network including a server and a plurality of client 
computers, the server and client computers each including means for transmitting 
data to and receiving data from an open network, 
wherein said computer software includes: 

applications level encryption and authentication software 
arranged to communicate with the server in order to: a.) mutually 
authenticate the server and the client computer initiating 
communications with the server and b.) generate a session key for 
use by the client computer initiating communications to encrypt files; 

and a shim arranged to intercept function calls and requests for 
service sent by an applications program to a lower level set of 
communications drivers in order to cause the applications level 
authentication and encryption program to communicate with the 
server, generate said session key, and use the session key operated 
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by the applications level encryption and authentication software to 
encrypt files sent by the applications program before transmittal over 
said open network. 

31 . (Amended) A method of carrying out communications over a multi-tier virtual 
private network, said network including a server and a plurality of client computers, 
the server and client computers each including means for transmitting data to and 
receiving data from an open network, comprising the steps of: 

intercepting function calls and requests for service sent by an applications 
program in one of said client computers to a lower level set of communications 
drivers; 

causing an applications level authentication and encryption program said one 
of said client computers to communicate with the server, generate [said] a session 
key, and use the se ssion kev generated by the applications level authentication and 
encryption program to encrypt files sent by the applications program before 
transmittal over said open network.--. 

REMARKS 

In parent U.S. Patent Application Ser. No. 08/917,341, claims 1, 5, 6, 16- 
19, 23, and 31 were rejected based on U.S. Patent No. 5,657,390 (Elgamel), and 
claims 2-4, 7-15, 20-22, 24-30, and 32-34 were allowed or indicated as being 
directed to allowable subject matter. In response, the claims indicated as allowable 
were re-written in independent form to include the limitations of the claim(s) from 
which they depended, with the remaining claims (including claims 1 6 and 1 7) being 
either already allowed or made dependent from allowed claims. 

The present continuation application is directed solely to the rejected claims. 
The reason for presenting the rejected claims in this continuation application is that 
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Applicant does not believe that the Elgamel patent discloses or suggests the 
following concepts: 

1. The concept of using "means for intercepting function calls... sent by an 
applications program on one of said client computers to a lower level set of 
communications drivers. ..and. ..means for causing an applications level 
authentication and encryption program. ..to. ..encrypt files sent by the applications 
program... ," as claimed in claim 1; 

2. A "shim" which intercepts the function calls and causes the applications 
level authentication and encryption program to communicate with the server and 
encrypt files as recited in claims 5 and 18; and 

3. A method corresponding to the "means" of claim 1 . 

In Elgamel, the security protocol is implemented through the use of a secure 
sockets layer (SSL) which is bound to the applications program. As explained in 
col. 5, lines 17-35, the sockets layer "establishes a sockets connection with an 
application running on a remote computer and then performs a security 
handshake." This is in contrast to a conventional socket layer which just 
establishes the sockets connection and does not provide authentication and 
encryption. 

The secure sockets layer disclosed in Elgamel thus replaces the conventional 
socket and is used by applications programs in the same manner as the 
conventional socket layer except that four additional function calls are added: 
"SSLopen, SSLwrite, SSL read, and SSLclose," as explained in col. 13, lines 1- 
57. In order to use the secure sockets layer to provide encryption services, an 
applications program must include the four function calls. 

In contrast, the present invention does not necessarily replace any existing 
sockets or libraries, or require modification of existing applications programs. 



Continuation of S.N. 08/917,341 

Instead, it intercepts function calls used by the existing non-secure socket and 
diverts them to an applications level authentication and encryption program, which 
then uses the existing socket to establish communications with an authentication 
proxy server in order to perform the authentication and generate session keys. The 
applications program making the function call could just as well be making the 
function call to the ordinary socket rather than to the shim and is not affected by 
the authentication and encryption that is taking place, and as a result the present 
invention can be used with a wider range of applications programs and with a 
wider range of operating systems and socket connections than is possible with an 
Elgamel-type secure sockets layer. This provides the unique advantage of enabling 
direct peer-to-peer communications by any applications capable of using whatever 
socket programs are already installed in the client. 

In other words, instead of just providing a socket that provides encryption 
services as in the Elgamel patent, the present invention inserts a shim between the 
sockets layer and applications programs that use the sockets layer. The shim 
diverts function calls to an applications level encryption and authentication program 
in a manner that is transparent to both the socket and the applications program, 
and the applications level encryption and authentication program initially directs 
communications to an authentication server in a manner which is also transparent 
to the applications program and sockets layer. There is no need to modify either 
the sockets layer or the applications program by adding new function calls as 
taught by Elgamel, and yet the invention provides a higher level of service and 
collateral functions for a wider variety of applications programs than is provided by 
the Elgamel secure sockets layer because, as claimed, authentication and 
generation of keys are carried out by communications between the applications 
level authentication and encryption program and a dedicated authentication proxy 
server. 
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As a result, it is believed that the claims presented in this continuation 
application are allowable over the Elgamel patent and all other references of record. 

Early and favorable consideration of the amended claims on the merits is 
respectfully requested. 



Respectfully submitted, 
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MULTI -ACCESS VIRTUAL PRIVATE NETWORK 



BACKGROUND OF THE INVENTION 



1 . Field of the Invention 
5 Thxs invention relates a system and method for 

allowing private communications over an open network/ and 
in particular to a virtual private network which provides 
data encryption and mutual authentication services for both 
client/ server and peer-to-peer applications at the 
10 applications, transport driver, and network driver levels. 



2 . Discussion of Related Art 

A virtual private network (VPN) is a system for 
securing communications between computers over an open 

15 network such as the Internet. By securing communications 
between the computers , the computers are linked together as 
if they were on a private local area network (LAN) , 
effectively extending the reach of the network to remote 
sites without the infrastructure costs of constructing a 

2 0 private network. As a result, physically separate LANs 



can work together as if they were a single LAN , remote 
computers can be temporarily connected to the LAN for 
communications with mobile workers or telecommuting, and 
electronic commerce can be carried out without the risks 
5 inherent in using an open network. 

In general, there are two approaches to virtual 
private networking, illustrated in Figs. 1A and IB. The 
first is to use a dedicated server 1, which may also 
function as a gateway to a secured network 2, to provide 

10 encryption and authentication services for establishment of 

secured links 3 between the server 1 and multiple clients 
4-6 over the open network 7, represented in Fig. 1A as a 
cloud, while the second is to permit private communications 
links 8 to be established between any two computers or 

15 computer systems 9-12 on network 7, as illustrated in Fig. 

IB. 

The advantages of a client/ server arrangement such as 
the one shown in Fig. 1A are that the server can handle 
functions requiring the majority of the computing 

2 0 resources, increasing the number of potential clients, and 

that management of the network, including key management is 
centralized. The disadvantage of a client/server network 
of this type is that peer-to-peer communications links 
between applications on the client computers cannot utilize 

25 the security and management functions provided by the 
server, leaving such communications unprotected. On the 
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other hand, the advantage of the direct peer-to-peer 
approach illustrated in Fig. IB is that it permits secured 
links to be established between any computers capable of 
carrying out the required security functions , with the 
5 disadvantages being the cost of configuring each computer 

to carry-out encryption, authentication, and key management 
functions, and the lack of central control. 

In both the client/server and peer-to-peer approaches, 
a virtual private network can in theory be based either on 

10 applications level technology or can operate at a lower 

level. Generally, however, peer-to-peer "tunneling" 
arrangements require modification of the lower layers of a 
computer's communications architecture, while client/server 
arrangements can use the applications level approach 

15 because less modification of the clients is required, and 
thus the two approaches are in practice mutually exclusive. 
The present invention, on the other hand, seeks to provide 
a virtual private network which utilizes a client/server 
approach, including centralized control of encryption, 

2 0 authentication, and key management functions, while at the 
same time enabling secured peer-to-peer communications 
between applications, by utilizing the server to provide 
authentication and session key generation functions for 
both client to server communications and peer-to-peer 

25 communications, providing a virtual private network capable 
of serving both as an extended intranet or wide area 
network (WAN), and as a commercial mass marketing network, 
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with high level mutual authentication and encryption 
provided for all communications . 

In order to completely integrate the two approaches 
and maximize the advantage of each approach, the invention 
5 maintains the applications level infrastructure of prior 
client server private networking arrangements, while adding 
shims to lower levels in order to accommodate a variety of 
peer-to-peer communications applications while utilizing 
the applications level infrastructure for authentication 
10 and session key generation purposes. This results in the 

synergistic effect that not only are existing peer-to-peer 
tunneling schemes and applications level client server 
security arrangements combined, but they are combined in a 
way which greatly reduces implementation costs 

15 In order to understand the present invention, it is 

necessary to understand a few basic concepts about computer 
to computer communications, including the concepts of 
"layers" and communications protocols, and of mutual 
authentication and file encryption. Further information 

20 about layers and protocols can be found in numerous sources 

available on the Internet, a few of which are listed at the 
end of this section, while a detailed description of a 
mutual authentication and encryption system and method 
suitable for use in connection with the present invention 

25 can be found in U.S. Patent No. 5,602,918, which is 
incorporated herein by reference. In general, the basic 
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communications protocols and architecture used by the 
present invention, as well as authentication, encryption, 
and key management schemes, are already well-known, and can 
be implemented as a matter of routine programming once the 
5 basic nature of the invention is understood. The changes 
made by the present invention to the conventional client 
server virtual private network may be thought of as, 
essentially, the addition of means, most conveniently 
implemented as shims, which add a secured mutual 
10 authentication and session key generation channel between 

the server and all parties to a communication, at all 
levels at which a communication can be carried out. 

Having explained the key differences between the 
present invention and existing systems, the basic concepts 

15 of layers and so forth will now be briefly explained by way 

of background. First, the concept of "layers," "tiers," 
and "levels," which essential to an understanding of the 
invention, simply refers to libraries or sets of software 
routines for carrying out a group of related functions, and 

20 which can conveniently be shared or called on by different 
programs at a higher level to facilitate programming, 
avoiding duplication and maximizing computer resources. 
For example, the Windows NT device driver architecture is 
made up of three basic layers, the first of which is the 

25 Network Driver Interface Specification (NDIS 3.0) layer, 
the second of which is called the Transport Driver 
Interface (TDI) layer, and the third being the file 
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systems. These layers are generically referred to as the 
network driver layer, the transport or transport driver 
layer, and the applications layer. 



In the windows NT architecture, the TDI layer formats 
5 data received from the various file systems or applications 
into packets or datagrams for transmission to a selected 
destination over the open network, while the NDIS layer 
controls the device drivers that send the data, packets, or 
IP datagrams , for example by converting the stream of data 
10 into a waveform suitable for transmission over a telephone 
line or a twisted pair cable of the type known as an 
Ethernet. 

By providing layers in this manner, an applications 
software programmer can design an application program to 

15 supply data to the TDI layer without having to re-program 

any of the specific functions carried out by that layer, 
and all of the transmission, verification, and other 
functions required to send a message will be taken care of 
the TDI layer without further involvement by the 

20 applications software. In a sense, each "layer" simply 

accepts data from the higher layer and formats it by adding 
a header or converting the data in a manner which is 
content independent, with retrieval of the data simply 
involving reverse conversion or stripping of the headers, 

25 the receiving software receiving the data as if the 
intervening layers did not exist. 
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In the case of Internet communications, the most 
commonly used set of software routines for the transport or 
TDI layer, which takes care of the data formatting and 
addressing, is the TCP/IP protocol, in which the transport 
5 control protocol (TCP) packages the data into datagrams and 
provides addressing, acknowledgements, and checksum 
functions, and the internet protocol (IP) further packages 
the TCP datagrams into packets by adding additional headers 
used in routing the packets to a destination address. 
10 Other transport protocols which can be included in the TDI 
layer include the user diagram protocol (UDP), the internet 
control message protocol (ICMP), and non-IP based protocols 
such as Netbeui or IPX. 

Additional 'protocols" are may be used at the 
15 applications level, although these protocols have nothing 
to do with the present invention except that they may be 
included in the applications programs served by the 
network. Common applications level protocols which utilize 
the TCP/IP protocol include hypertext transfer protocol 
20 (HTTP), simple mail transfer protocol (SMTP), and file 

transfer protocol (FTP), all of which operate at the layer 
above the transport layer. 

Some applications are written to directly call upon 
the TCP functions. However, for most applications 
25 utilizing a graphical user interface conveniently rely on 
a set of software routines which are considered to operate 
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above the TDI layer, and are known as sockets. Sockets 
serve as an interface between the TCP set of functions, or 
stack, and various applications, by providing libraries of 
routines which facilitate TCP function calls, so that the 
5 application simply has to refer to the socket library in 
order to carry out the appropriate function calls. For 
Windows applications, a commonly used non-proprietary 
socket is the Windows socket, known as Winsock, although 
sockets exist for other operating systems or platforms, and 
10 alternative sockets are also available for Windows, 
including the winsock 2 socket currently under development. 

In order to implement a virtual private network, the 
encryption and authentication functions must be carried out 
at one of the above "levels," for example by modifying the 

15 network drivers to encrypt the IP datagrams, by inserting 
authentication headers into the TCP/IP stacks, or by 
writing applications to perform these functions using the 
existing drivers. If possible, it is generally desirable 
to minimize modification of the existing levels by adding 

20 a layer to perform the desired functions, calling upon the 
services of the layer below, while utilizing the same 
function calls so that the higher layer also does not need 
to be modified. Such a layer is commonly referred to as a 
"shim. " 



25 As indicated above, the preferred approach to 

implementing client/server virtual private networks is to 



use an applications level security system to encrypt files 
to be transmitted, and to then utilize existing 
communications layers such as Winsock, or TCP/IP directly. 
This is the approach taken by the commercially available 
5 access control system known as SmartGATE™, developed by V- 

One Corp. of Germantown, Md., which provides both 
encryption and mutual authentication at the applications 
level utilizing a dedicated server known as an 
authentication server and authentication client software 

10 installed at the applications level on the client 

computers. A description of the manner in which encryption 
and mutual authentication is carried out may be found in 
the above-cited U.S. Patent No. 5,602,918. While the 
principles of the invention are applicable to other 

15 client/server based virtual private networks, SmartGATE™ is 
used as an example because it provides the most complete 
range of mutual authentication and encryption services 
currently available. 

The present invention can be implemented using the 
2 0 existing SmartGATE™ system, but adds mutual authentication 

and encryption services to lower layers by intercepting 
function calls or data packets and, during initialization 
of a communications link, establishing separate channels 
between the party initiating the communication and the 
25 authentication server, and between the authentication 
server and the party which is to share in the 
communication, so as to mutually authenticate the parties 
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with respect to the server, and so as to establish a 
session key which can be used for further direct 
communications between the parties . 



A number of protocols exist which can be used, in 
5 total or in part, to implement the mutual authentication 
and encryption services at the lower layers, using the same 
basic authentication and encryption scheme currently 
implemented by SmartGATE™ at the applications level. These 
include, by way of example, the SOCKS protocol, which 
10 places a shim between the TDI or transport layer and the 
applications , and the commercially available program, known 
as SnareNet, which operates at the network driver level and 
can be directly utilized in connection with the present 
invention . 

15 On the other hand, a network level implementation such 

as the SKIP protocol, which operates below the TDI layer to 
encrypt the datagrams, and which in its description 
explicitly precludes the generation of session keys (see 
the above cited U.S. Patent No. 5,602,918), is 

2 0 fundamentally different in concept than the present 

invention. Similarly, alternative implementations such as 
Point-to-Point Tunneling Protocol (PPTP) which involve 
modifying the TCP/IP stack and/or hardware to provide 
encryption, as opposed to inserting shims, are not utilized 

25 by the preferred embodiment of the present invention, 
although individual aspects of the protocol could perhaps 
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be used, and the present system could be added to computers 
also configured to accept PPTP communications . 

The SmartGATE™ system uses public key and DES 
encryption to provide two-way authentication and 56-bit 
5 encrypted communications between a server equipped with the 

SmartGATE program and client computers equipped with a 
separate program. Currently, SmartGATE™ operates at the 
highest level, or applications level, by using shared 
secret keys to generate a session key for use in further 

10 communications between the authentication server or gateway 
and the client program. Since the session key depends on 
the secret keys at the gateway and client sides of the 
communication, mutual authentication is established during 
generation of the session key, which can then be used to 

15 encrypt further communications . 

When installed on a client system, the SmartGATE™ 
client software reads a request for communications by an 
applications program, such as a browser program, and then 
proceeds to establish its own communications link with the 

20 destination server to determine if the server is an 
authentication server. if it is not, control of 

communications is relinquished, but if it is, then the 
security program and the server carry out a 
challenge/response routine in order to generate the session 

2 5 key, and all further communications are encrypted by the 
security program. Although this program is placed between 
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the Winsock layer and the applications, it does not 
function as a shim, however, because it only affects 
communications directed to the authentication server. 

Having briefly summarized the concepts used by the 
present invention, including the concepts of layers, 
protocols, and shims, and having described a specific 
applications level security program which is to be modified 
according to the present invention by adding shims in a way 
which enables secured authentication and session key 
generation channels to be set up from the lower layers, it 
should now be possible to understand the nature of the 
invention, and in particular how it integrates the two 
approaches to virtual private networking in a way which 
greatly expands the concept and yet can easily be 
implemented. More details will be given below, but as a 
final observation in this background portion of the patent 
specification, it should be noted that while the overall 
concept of the invention is in a sense very simple, it is 
fundamentally at odds with present approaches. For 
example, the literature is replete with references to 
conflicts between VPN standards and implementations, as 
exemplified by the title of an article from LAN Times On- 
Line, 9/96, (http://www.wcmh.com/), which reads Clash Over 
VPN Supremacy. Even a cursory search of the available 
literature indicates that the amount of information and 
choices available to those wishing to set up a virtual 
private network is overwhelming. One can choose between 
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Netscape Communications Secure Socket Layer, Open Market 
Inc.'s Secure HTTP, Microsoft's PPTP, among others. 
However, all of these approaches operate at a single level, 
and force a choice between establishing a network of the 
type shown m Fig. 1A and a network of the type shown in 
Fig. IB. Only the present invention offer the advantages 
of both approaches, without the inflexibility of 
client/server arrangements or the costs of more distributed 
architectures . 

For further information on the various competing VPN 
protocols and systems, see also The Development of Network 
Security Technologies, Internet Smartsec, 2/97 
(http://www.smartsec.se), which compares SmartGATE™ to 
other application level security systems, including PPTP, 
SSL, and S-HTTP; Point-To-Point Tunneling Protocol (PPTP) 
Frequently Asked Questions, Microsoft Corp., date unknown, 
(http://www.microsoft.com), Simple Key-Management for 
Internet Protocols (SKIP), Aziz et al . , date unknown, 
(http://skip.incog.com), and SOCKS Protocol Version 5, RFC 
1928, Leech et al . , 3/96 (http://andrew2.andrew.cmu.edu) 
(this document describes a protocol involving a TDI shim) . 
For more general information on security problems, Internet 
protocols, and sockets, see Introduction to the Internet 
Protocols, Charles L. Hedrick, Rutgers University, 1987 
(http://oac3.hsc.uth.tmc.edu); Windows Sockets - Where 
Necessity is the Mother of Reinvention, Stardust 
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Technologies, Inc., 1996, ( http/ /www. Stardust .com) , and 
Secure Internet Connections , LAN Times, 6/17/96 (Ibid). 



SUMMARY OF THE INVENTION 

It is accordingly a principal objective of the 
5 invention to provide a client/server virtual private 
network which is capable not only of carrying out 
authenticated secure communications over an open network 
between an authentication server and clients, but also 
authenticated secure peer-to-peer communications. 

10 It is also an objective the invention to provide a 

virtual private network that provides data encryption and 
mutual authentication for both client/server and peer-to- 
peer communications for different-types of applications, 
using both the applications level and lower levels of a 

15 communications hierarchy. 

It is a further objective of the invention to provide 
a client/ server virtual private network which can provide 
both client/ server and peer-to-peer encryption and 
authentication services for any application sharing a 
2 0 specified socket or sockets, whether or not the application 
is recognized by the encryption and authentication program. 

It is a still further objective of the invention to 
provide a client/server virtual private network which can 

14 



provide encryption and authentication services at the 
applications level, transport driver interface level, and 
network interface level, without the need for modifying 
either the communication driver or network, driver, or any 
5 sockets utilizing the communications driver interface. 

It is yet another objective of the invention to 
provide a virtual private network which provides encryption 
and authentication services for peer-to-peer communications 
while maintaining centralized control of key distribution 
10 and management functions. 

Finally, it is also an objective of the invention to 
provide a virtual private network which provides encryption 
and authentication services for peer-to-peer communications 
and in which registration is carried out by a central 
15 gateway server. 

These objectives of the invention are accomplished by 
providing a virtual private network for communicating 
between a server and clients over an open network and in 
which the clients are equipped with an applications level 

2 0 encryption and mutual authentication program which includes 
at least one shim positioned above either the socket, 
transport driver interface, or network interface layers of 
a client computers communications hierarchy, and which 
intercepts function calls or data packets in order to 

25 authenticate the parties to the communication by 
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establishing secured channels between the server and the 
parties to the communication, prior to establishment of the 
secured communications link between the parties, in order 
to carry out mutual authentication and session key 
5 generation functions. 

More particularly, according to the principles of a 
preferred embodiment of the invention, client 
communications software is provided which, at the socket or 
transport driver interface levels, intercepts function 

10 calls to the socket or transport driver and directs calls 

to the authentication server in order to perform encryption 
and authentication routines , and at the network driver 
interface, performs encryption and authentication functions 
by intercepting the datagrams or data portions of the 

15 packets transmitted by the transport driver interface based 

on communications between the authentication server and the 
client. According to this aspect of the invention, a 
system of providing authentication and encryption services 
for the purpose of establishing a virtual private network 

2 0 includes a plurality of shims arranged to operate at 
different protocol levels in order to establish a common 
secure communications link to an authentication server. 

In one especially preferred embodiment of the 
invention, the client software includes a Winsock shim 
25 arranged to intercept function calls to the Winsock library 
on a client machine and redirect initial communications 
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through the authentication client software to the 
authentication server, so that any function calls to the 
Winsock library of programs are intercepted by the shim and 
carried out by the applications level security program. In 
this embodiment, the client authentication software 
substitutes its own function calls for the original 
function calls in order to establish a secured 
communications link to the authentication server over which 
such functions as mutual authentication between the client 
and server, indirect authentication of peer applications by 
the now trusted server, session key generation, are carried 
out, as well as ancillary functions such as on-line 
registration (OLR), utilizing the unmodified original 
Winsock library and TCP/IP communications stacks. 

By inserting a shim at the Winsock level, an 
applications level client/ server based security program 
such as SmartGATE™ can be used to provide secure 
communications for any application which utilizes the 
Winsock library, in addition, by including analogous shims 
at other levels, the invention can be used to secure 
virtually any communications application, including those 
which by-pass the TDI layer and communicate directly with 
the network driver level. 

Instead of the current array of mutually exclusive 
alternative methods and systems of establishing secured 
communications over an open network, the invention thus 
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provides a single integrated method and system capable of 
carrying out both client/ server communications and peer-to- 
peer communications between a wide variety of 
communications applications regardless of whether the 
applications use a socket or even commonly accepted 
internet protocols, with complete mutual authentication and 
encryption of data files at all levels and between all 
parties to the network. 

It will be appreciated that the term "virtual private 
network" is not to be taken as limiting, and that the 
principles of the invention can be applied to any remote 
access schemes which utilize the Internet or other 
relatively insecure networks to provide access for remote 
users, corporate intranets, and electronic commerce. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1A is a schematic diagram of a client/server 
virtual private network. 

Fig. IB is a schematic diagram of an alternative 
virtual private network based on peer-to-peer 
communications . 

Fig. 2 is a functional block diagram showing the 
operation of an applications level security program in a 
conventional communications network hierarchy. 
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Fig. 3 is a functional block diagram showing the 
communications network hierarchy of Fig. 1, modified to 
provide a second layer of service in accordance with the 
principles of a preferred embodiment of the invention. 

Fig. 4 is a functional block diagram showing the 
communications network hierarchy of Fig. 2, modified to 
provide a third layer of service in accordance with the 
principles of the preferred embodiment. 

Fig. 5 is a functional block diagram showing the 
communication network hierarchy of Fig. 3, modified to 
provide a fourth layer of service in accordance with the 
principles of the preferred embodiment. 

Fig . 6 is a schematic diagram of a virtual private 
network utilizing the principles of the preferred 
embodiment of the invention . 

Fig. 7 is a flowchart illustrating a method of 
implementing the system of the preferred embodiment. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 2 illustrates the operation of a client 
authentication program which is utilized in the present 
invention. An example of such a program is the SmartGATE™ 
program discussed briefly above, although other 
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applications level security programs, whether or not token 
based, could be modified in a manner similar to that 
discussed in the following description. The illustrated 
hierarchy is the windows NT architecture, although versions 
5 of SmartGATE™ exist for other architectures, and the 
invention could easily be adapted for use with any version 
of SmartGATE™, including UNIX and Macintosh versions, as 
well as for use with applications level security programs 
designed for communications architectures other than those 

10 supported by SmartGATE™ . Conversely, it is intended that 
the present invention can be used with authentication and 
encryption schemes other than that used by SmartGATE™ and 
disclosed in U.S. Patent No. 5,602,918. For purposes of 
convenxence, therefore, the software represented by 

15 SmartGATE™ is simply referred to as client authentication 

software . 

In addition, it noted that the client computer 
architectures illustrated in Figs. 3-6, which are modified 
versions of the architecture of Fig. 2, is to be used with 

2 0 an overall network layout such as the one illustrated in 
Fig. 6, which includes an authentication server that may be 
a SmartGATE™ server, or another server depending on the 
client authentication software. The invention is not 
merely the addition of shims to the client software, but 

25 involves the manner in which the shims are used in the 
establishment of the authentications and key generation 
links to the server. 
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Turning to Fig. 2, which provides background for the 
description of the invention illustrated in Figs. 3-6, the 
client authentication software 2 0 is situated above the 
boundary of the transport or TDI layer 21 and is designed 
5 to utilize a socket 22, such as Winsock, to carry out 
communications with the authentication server 23 shown in 
Fig. 6 by means of a transport protocol such as TCP/IP, 
UDP, or the like, which in turn supply datagrams or packets 
to a hardware driver layer 24, such as NDIS 3.0, of a 
10 network or modem connection 25. 

In operation, the client authentication software 20 
intercepts interconnect calls 2 6 form client authentication 
software supported applications 2 7 and, if the calls are 
directed to the authentication server 23, or to a server 28 

15 situated on a secured network whose access is controlled by 
the authentication server, establishes a secured 
communications link to the server by executing appropriate 
function calls 2 9 to the socket library, which in turn 
transmits function calls 3 0 to the TDI layer, causing the 

20 TDI layer to form datagrams or packets 31. Datagrams or 
packets 31 are then formatted over packaged for 
transmission by the hardware drivers 24 and sent to the 
communications network in the form of Ethernet packets or 
analog signals 32 containing the original datagrams from 

25 the TDI layer. Once the secured communications link has 
been established, client authentication software 20 
encrypts all further data communications 34 from 



21 



applications 27, which are indicated by dashed lines, 
before handing them off to the next lower layer in the form 
of encrypted files 35. The dashed lines are shown in Fig. 
2 as extending only to the TDI layer 21, because the 
5 datagrams formed by the TDI layer are indistinguishable as 
to content, but it is to be understood that datagrams or 
packets 31 carry both the communications used to establish 
the secure channel, and the encrypted files subsequently 
sent therethrough. 

10 Finally, in the case of SmartGATE™, the 

authentication client software utilizes either a smart card 
or secured file to supply the secret keys used during 
authentication to generate a session key for encryption of 
further communications, and also to carry out certain other 

15 encryption and authentication functions , although it is of 
course within the scope of the invention to use key 
distribution and authentication methods which do not rely 
on smartcards or tokens , and the tokens are not involved in 
any of the basic communications functions of the client 

2 0 authentication software 20. 

In addition to the applications 2 7 which communicate 
with the server via the authentication/encryption software 
20, a typical system will have a number of additional 
software applications 36 and 3 7 capable of carrying out 
25 communications over the open network, but which the 
authentication client software is not configured to handle, 
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and which are not specifically adapted or intended to carry 
out communications with the authentication server. These 
are referred to herein as peer-to-peer applications, and 
can include applications which use the same sockets as the 
5 authentication client software, applications which directly 

call upon a transport driver interface stack, whether using 
the same protocol as the authentication client software or 
another protocol, all of which are intended to be 
represented by the TDI layer, and applications which are 

10 written to call directly upon the hardware drivers. These 
peer-to-peer applications may have their own encryption and 
authentication capabilities, but cannot utilize the 
services of the authentication server or client software, 
and therefore the function calls made by the applications 

15 and the files transmitted are indicated by separate 

reference numerals 4 0-4 3. 

It will be appreciated by those skilled in the art 
that lower layer application programs which generate 
packets in forms other than those represented by the TDI 

20 layer are also possible, and should be considered within 
the scope of the invention, but at present virtually all 
open network applications use at least one of the TDI 
protocols, and thus while these programs may interact 
directly with the network driver layer, and require a 

25 network driver layer shim, as will be discussed below, are 

illustrated for purposes of convenience as part of the TDI 
layer applications . 
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Turning now to a preferred embodiment of the 
invention, the arrangement shown in Fig. 3 modifies the 
arrangement of Fig. 2 by adding a socket shim 5 0 between 
the socket 22 utilized by the authentication client 
5 software 2 0, the peer-to-peer applications 3 6 which also 
utilize the socket 20, and the authentication client 
software itself. The shim 5 0 operates by hooking or 
intercepting call initiation function calls 40 made to the 
socket and, in response thereto, having the authentication 

10 client software initiate communications with the 
authentication server 23, shown in Fig. 6, in order to 
carry out the authentication protocol, as will be discussed 
in more detail below. Shim 5 0 also causes files 41 
intended for the TDI layer to be diverted to the 

15 authentication software for encryption based on the session 
keys generated during the initial communications with the 
authentication server, and transmission as encrypted files 
51 addressed to the peer application, also shown in Fig. 6, 
which could also be an application on the application 

20 server 28. 

Since the basic authentication client software is 
designed to send all communications directly to the 
authentication server, while the peer-to-peer applications 
are designed only to communicate with "peers" 45 and not 
25 with the authentication server, the principal function of 
shim 5 0 is to arrange for the destination of address of the 
communication to be supplied to both the authentication 
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client software and to authentication server, even though 
the peer application assumes that it is communicating only 
with the peer application. This function permits session 
key encrypted communications to be forwarded directly to 
5 the peer application, as illustrated in Fig. 6, while the 
latter function provides the authentication server with the 
client address so that the authentication server can 
establish a secured and authenticated link with the peer 
application, via authentication client software on the peer 
10 computer, and transmit the session key to the peer 
application or at least enable the peer application to 
recreate the session so that it can decrypt the encrypted 
files received directly from the client application. 

Thus, while it is appreciated that the use of socket 
15 shims is well-known, as mentioned above, the socket shim 
shown in Fig. 2 has the unique function of enabling direct 
peer-to-peer communications with mediation by the 
authentication server, permitting the highest level of 
authentication service and collateral functions. In 
2 0 addition, because of the mediation by the key server, the 
peer applications do not need to have a shared secret key, 
allowing centralized key management, with only the 
authentication server having access to all of the client's 
secret keys . 

2 5 Figs. 4 shows the variation of the client 

authentication software 2 0 in which a TDI shim 52 similar 
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in function to the socket shim 5 0 is provided above the TDI 
layer. Like the socket shim, implementation of the TDI 
shim essentially simply involves diverting certain 
information to the client software in order to establish a 
5 communications link with the authentication server, and 
subsequently perform encryption to obtain encrypted files 
54 for transmission directly through the TDI layer in the 
usual manner. As with the socket shim, TDI shims are not 
new and can be implemented in known manner, by intercepting 
10 TDI service requests, but with the difference from prior 
TDI shims that the TDI shim works with the authentication 
software 2 0 and authentication server to authenticate 
communications and generate a session key. 



Finally, as shown in Fig. 5, a further layer of 
15 authentication and encryption may be added by adding a 
network driver shim 55, either to the arrangement shown in 
Fig. 3 without the TDI shim, in combination with the TDI 
shim shown in Fig. 4, or in combination with the TDI shim 
of Fig. 4 but not the socket shim, to provide for 
2 0 authentication of communications at the network driver 
layer. At this layer, the shim 55 intercepts IP packets 
from applications 56, but instead of referring back to the 
applications level routine, checks the destination address 
(which can be in TCP format, UDP format, and so forth), 
25 establishes a session key by communications with the 
authentication server, converts the session key into a 
format which can be used to encrypt the IP packet, and 
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sends the IP packet towards the destination, all by- 
carrying out the necessary operations at the network driver 
level, in a manner similar to that utilized by the above- 
mentioned SnareNet software program, but with the 
5 difference that the authenticating communications link and 

key generation is carried out by packets addressed to a 
corresponding layer 5 6 of the authentication server, which 
may be further connected to an applications server 57. 

It will be noted that since the IP packets are not 
10 distinguishable by content, the network driver layer shim 

could be used as an additional level of security, rather 
than as an alternative to applications level encryption, 
with the encrypted files generated by software 20 being 
further encrypted by shim 5 5 before transmission to the 
15 authentication server or associated gateway. 

The overall system utilizing the authentication client 
software illustrated in Figs. 3-5 is schematically 
illustrated in Fig. 6. The principal components of the 
overall system are the client computers containing software 

2 0 of the type illustrated in Figs. 2-5, including client 
authentication software 2 0 and shims 50, 53, and/or 55, and 
applications with communications capabilities (represented 
by applications 27, 36, 37, and 5 6 on one client, and 
application 45 on the other). For purposes of 

25 illustration, the client of Figs. 6 is thus depicted as 
including applications for communicating at the highest 
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levels, such as the SmartGATE™ proxy application, 
applications for communicating at the network driver level 
with corresponding applications connected to the lower 
layer of the authentication server, and peer-to-peer 
5 applications with no capability of communicating with 

SmartGATE™, but which use sockets or TDI protocols 
recognized by the shims. 

In the case of the SmartGATE™ proxy application, 
communications are established in the same manner as in the 

10 currently available version of the SmartGATE™ 
authentication client software, and as described in U.S. 
Patent No. 5,602,918, the communications link being 
indicated by arrows 6 0 and 61, with arrow 60 representing 
the client/ server response channel used to authenticate the 

15 parties and generate the session key. 

In the case of a peer-to-peer application, in which 
the clients wish to communicate over a direct link 62, the 
invention provides for the function calls establishing the 
communications to be intercepted and the initialization 

20 procedure routed through channel 61 to the authentication 
server 23. Server 23 then opens a secured channel 63 to 
the authentication client software 2 0 associated with peer 
application 45 by performing the same mutual authentication 
procedure performed for the purpose of establishing channel 

25 63, and once the channel is established with its own 
session key, transmits information using the channel 63 
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session key which allows the client to recreate the channel 
60 session key for use in decrypting communications sent 
over channel 62. Alternatively, after establishing channel 
63, the channel 60 session key could be used to transmit 
5 back to the original sending party information necessary to 
recreate the channel 63 session key. In either case, the 
authentication server is thus used to establish a fully 
authenticated " tunnel" between the peer applications 
without the need to modify any of the sockets , TDI 

10 protocols, or hardware drivers on either of the client 
computers. While the transmitting peer application has no 
way of directly authenticating the receiving peer, only a 
receiving peer authenticated by the authentication server 
will be able to generate the necessary session keys, and 

15 thus each of the parties to the communication is 
effectively authenticated. 

For the lower layer application 56, a similar protocol 
may be employed, in which the attempted communication 
between lower layer applications is intercepted, and the 

20 communications link to the authentication server is used to 
generate a session key, which is then used to encrypt the 
packets or datagrams being sent. In this case, the 
destination must be the lower layer of the authentication 
server, and thus the communications link is indicated by a 

25 separate channel 67. 
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Finally, the procedures associated with the network 
illustrated in Fig. 6 are summarized in the flowchart of 
Fig. 7. For communications directly with the applications 
level portion of the server 23, steps 100-103 are used, 
5 while for peer-to-peer communications, steps 104-109 are 
used, and for network driver level communications, steps 
110-114 are used. 

In particular, step 100 by which the applications 
level authentication program 2 0 illustrated in Figs. 3-5 
receives a call initiation request, either directly from a 
supported applications program 2 7 or from a programs 36 and 
37 via one of the shims 50 and 53, step 101 is step by 
which the program 2 0 addresses the authentication server, 
step 102 is the step by which the client and server are 
mutually authenticated and the session keys generated 
using, for example, the procedure described in U.S. Patent 
No. 5, 602 , 918, and step 103 is the step by which program 20 
encrypts further communications received directly or via 
shims 5 0 and 5 3 from the applications programs 27, 36, and 
37. 

For peer-to-peer communications, step 105, which is 
part of step 100, is the step by which the peer address is 
supplied to program 20, steps 106 and 107 are identical to 
steps 101 and 102, step 108 is the step by which 
25 communications channel 63 shown in Figure 6 is established, 

step 109 is the step by which the destination computer 

30 
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authenticated by the server is enabled to decrypt 
communications received over channel 62, and step 110 is 
the step by which program 2 0 encrypts the communications . 
It will of course be appreciated that these steps represent 
5 only a summary of the steps involved in carrying out the 

present invention, and that further steps will be apparent 
to those skilled in the art based on the above description 
of the apparatus and software portions of the preferred 
embodiment of the invention. 



10 Having thus described various preferred embodiments of 

the invention, those skilled in the art will appreciate 
that variations and modifications of the preferred 
embodiment may be made without departing from the scope of 
the invention. It is accordingly intended that the 

15 invention not be limited by the above description or 
accompanying drawings, but that it be defined solely in 
accordance with the appended claims. 
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I claim: 



1. Apparatus for carrying out communications over a 
multi-tier virtual private network, said network including 
a server and a plurality of client computers, the server 
and client computers each including means for transmitting 
data to and receiving data from an open network, 
comprising : 

means for intercepting function calls and requests for 
service sent by an applications program on one of said 
client computers to a lower level set of communications 
drivers; and 

means for causing an applications level authentication 
and encryption program in said one of said client computers 
to communicate with the server, generate said session key, 
and encrypt files sent by the applications program before 
transmittal over said open network. 

2. Apparatus as claimed in claim 1, further comprising 
means for intercepting files packaged by a transport driver 
interface layer to form packets and encrypting the packets 
using a session key generated during communications with a 
lower layer of the server. 

3. A method as claimed in claim 1, further comprising 
means for intercepting a destination address during 

initialization of communications between said one of said 
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client computers and a second of said client computers on 
said virtual private network; 

means for causing said applications level 
authentication and encryption program to communicate with 
the server to carry out functions a.) and b.); 

means for transmitting said destination address to 
said server; 

means for causing said server to carry-out functions 
a.) and b.) with respect to the second of said two client 
computers ; 

means for enabling said second of said two client 
computers to recreate the session key; 

means for causing said authentication software to 
encrypt files to be sent to the destination address using 
the session key; and 

means for transmitting the encrypted files directly to 
the destination address. 

4. Apparatus as claimed in claim 3, wherein said means 
for intercepting the destination address is carried out by 
a shim positioned between a peer-to-peer applications 
program and a layer of a communications driver architecture 
of said one of the two client computers. 

5. A multi-tier virtual private network, comprising: 

a server and a plurality of client computers, the 
server and client computers each including means for 
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transmitting data to and receiving data from an open 
network, 

wherein said means for transmitting data to and 
receiving data from the open network includes, in any 
client computer initiating communications with the server: 
applications level encryption and 
authentication software arranged to communicate 
with the server in order to: a.) mutually 
authenticate the server and the client computer 
initiating communications with the server and b.) 
generate a session key for use by the client 
computer initiating communications to encrypt 
files; 

at least one lower level set of 
communications drivers; 

and a shim arranged to intercept function 
calls and requests for service sent by an 
applications program to the lower level set of 
communications drivers in order to cause the 
applications level authentication and encryption 
program to communicate with the server, generate 
said session key, and encrypt files sent by the 
applications program before transmittal over said 
open network . 

6. A multi-tier virtual private network as claimed in 
claim 5 , wherein said lower level set of communications 
drivers includes a network driver layer, a transport driver 
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interface layer arranged to package applications files as 
packets capable of being routed over the open network and 
supply the packets to the network driver layer for 
transmission to the open network, and an applications 
socket for facilitating service requests by said 
applications program to the transport driver interface 
layer, and wherein said shim is a socket shim positioned 
between the applications program and the socket to 
intercept function calls to the socket in order to cause 
the applications level authentication and encryption 
program to communicate with the server, generate said 
session key, and encrypt files sent by the applications 
program before the files are packaged by the transport 
driver interface layer. 

7 . A multi-tier virtual private network as claimed in 
claim 6, wherein said applications program is a peer-to- 
peer communications program, and wherein a peer application 
destination address, included in said function calls to the 
socket, is diverted by the socket shim and wherein a 
destination address including said intercepted function 
calls is supplied to the server during communications with 
the server, causing the service to establish a 
communications link with a peer application, mutually 
authenticate the peer application, and enable the peer 
application to reconstruct the session key in order to 
receive encrypted files sent by the peer-to-peer 
communications program over the open network. 
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8. A multi-tier virtual private network as claimed in 
claim 6, further including a transport driver interface 
shim positioned between the transport driver interface 
layer and a second applications program, for intercepting 
requests from the second applications program for service 
by the transport driver interface layer in order to cause 
the applications level authentication and encryption 
program to communicate with the server, generate said 
session key, and encrypt files sent by the applications 
program before the files are packaged by the transport 
driver interface layer. 

9. A multi-tier virtual private network as claimed in 
claim 8, furtner comprising a network driver layer shim 
positioned between the network driver layer and the 
transport driver interface layer and arranged to intercept 
files packaged by the transport driver interface layer and 
encrypt the files using a session key generated during 
communications with a lower layer of the server. 

10. A multi-tier virtual private network as claimed in 
claim 5 , wherein said lower level set of communications 
drivers includes a network driver layer, and a transport 
driver interface layer arranged to package applications 
files as packets capable of being routed over the open 
network and supply the packets to the network driver layer 
for transmission to the open network, and wherein said shim 
is a transport driver interface layer shim positioned 
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between the applications program and the transport driver 
interface layer to intercept service requests by the 
applications program to the transport driver interface 
layer in order to cause the applications level 
authentication and encryption program to communicate with 
the server, generate said session key, and encrypt files 
sent by the applications program before the files are 
packaged by the transport driver interface layer. 

11. A multi-tier virtual private network as claimed in 
claim 10, wherein said applications program is a peer-to- 
peer communications program, and wherein a peer application 
destination address, included in said intercepted requests 
for service, is diverted by the transport driver interface 
layer shim and supplied to the server during communications 
with the server, causing the service to establish a 
communications link with a peer application, mutually 
authenticate the peer application, and enable the peer 
application to reconstruct the session key in order to 
receive encrypted files sent by the peer-to-peer 
communications program over the open network. 

12 . A multi-tier virtual private network as claimed in 
claim 10, further comprising a network driver layer shim 
positioned between the network driver layer and the 
transport driver interface layer and arranged to intercept 
files packaged by the transport driver interface layer and 
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encrypt the files using a session key generated during 
communications with a lower layer of the server. 

13. A multi-tier virtual private network, comprising: 

a server and a plurality of client computers, the 
server and client computers each including means for 
transmitting data to and receiving data from an open 
network, 

wherein said means for transmitting data to and 
receiving data from the open network includes, in any 
client computer initiating communications with the server: 
applications level encryption and 
authentication software arranged to communicate 
with the server in order to: a.) mutually 
authenticate the server and the client computer 
initiating communications with the server and b. ) 
generate a session key for use by the client 
computer initiating communications to encrypt 
files; and 

at least one lower level set of 
communications drivers, 

wherein said lower level set of 
communications drivers includes a network driver 
layer, a transport driver interface layer 
arranged to package applications files as packets 
capable of being routed over the open network and 
supply the packets to the network driver layer 
for transmission to the open network, and a 
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network driver layer shim positioned between the 
transport driver interface layer and the network 
driver layer and arranged to intercept files 
packaged by the transport driver interface layer 
and encrypt the files using a session key 
generated during communications with a lower 
layer of the server. 

14. A multi-tier virtual private network, comprising: 

a server and a plurality of client computers, the 
server and client computers each including means for 
transmitting data to and receiving data from an open 
network, 

wherein said means for transmitting data to and 
receiving data from the open network includes, in any 
client computer initiating communications with the server: 
applications level encryption and 
authentication software arranged to communicate 
with the server in order to: a. ) mutually 
authenticate the server and the client computer 
initiating communications with the server and b.) 
generate a session key for use by the client 
computer initiating communications to encrypt 
files; and 

further comprising means for securing peer-to-peer 
communications between applications on two of said client 
computers, said peer-to-peer communications securing means 
comprising: 
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means for intercepting a destination address 
during initialization of communications by a 
first of said two client computers; 

means for causing said authentication 
software to communicate with the server to carry 
out functions a.) and b.); 

means for transmitting said destination 
address to said server; 

means for causing said server to carry-out 
functions a.) and b.) with respect to the second 
of said two client computers; 

means for enabling said second of said two 
client computers to recreate the session key; 

means for causing said authentication 
software to encrypt files to be sent to the 
destination address using the session key; 

means for transmitting the encrypted files 
directly to the destination address. 

15. A multi-tier virtual private network as claimed in 
claim 14, wherein said means for intercepting the 
destination address comprises a shim positioned between the 
peer-to-peer applications program and a layer of a 
communications driver architecture of said first of the two 
client computers . 

16. A multi-tier virtual private network as claimed in 
claim 5, wherein said shim is positioned above a socket, 
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the socket being positioned above a transport driver layer 
of said communications driver architecture. 

17. A multi-tier virtual private network as claimed in 
claim 5, wherein said shim is positioned above a transport 
driver layer of said communications driver architecture. 

18. Computer software for installation on a client 
computer of a multi-tier virtual private network, said 
network including a server and a plurality of client 
computers, the server and client computers each including 
means for transmitting data to and receiving data from an 
open network, 

wherein said computer software includes: 

applications level encryption and 
authentication software arranged to communicate 
with the server in order to: a. ) mutually 
authenticate the server and the client computer 
initiating communications with the server and b. ) 
generate a session key for use by the client 
computer initiating communications to encrypt 
files ; 

and a shim arranged to intercept function 
calls and reguests for service sent by an 
applications program to a lower level set of 
communications drivers in order to cause the 
applications level authentication and encryption 
program to communicate with the server, generate 
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said session key, and encrypt files sent by the 
applications program before transmittal over said 
open network. 

19. Computer software as claimed in claim 18, wherein said 
lower level set of communications drivers includes a 
network driver layer, a transport driver interface layer 
arranged to package applications files as packets capable 
of being routed over the open network and supply the 
packets to the network driver layer for transmission to the 
open network, and an applications socket for facilitating 
service requests by said applications program to the 
transport driver interface layer, and wherein said shim is 
a socket shim positioned between the applications program 
and the socket to intercept function calls to the socket in 
order to cause the applications level authentication and 
encryption program to communicate with the server, generate 
said session key, and encrypt files sent by the 
applications program before the files are packaged by the 
transport driver interface layer. 

20. Computer software as claimed in claim 19, wherein said 
applications program is a peer-to-peer communications 
program, and wherein a peer application destination 
address, included in said function calls to the socket, is 
diverted by the socket shim and wherein a destination 
address including said intercepted function calls is 
supplied to the server during communications with the 
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server, causing the service to establish a communications 
link with a peer application, mutually authenticate the 
peer application, and enable the peer application to 
reconstruct the session key in order to receive encrypted 
files sent by the peer-to-peer communications program over 
the open network. 

21. Computer software as claimed in claim 19, further 
including a transport driver interface shim positioned 
between the transport driver interface layer and a second 
applications program, for intercepting requests from the 
second applications program for service by the transport 
driver interface layer in order to cause the applications 
level authentication and encryption program to communicate 
with the server, generate said session key, and encrypt 
files sent by the applications program before the files are 
packaged by the transport driver interface layer. 

22. Computer software as claimed in claim 21, further 
comprising a network driver layer shim positioned between 
the network driver layer and the transport driver interface 
layer and arranged to intercept files packaged by the 
transport driver interface layer and encrypt the files 
using a session key generated during communications with a 
lower layer of the server. 

23. Computer software as claimed in claim 18, wherein said 
lower level set of communications drivers includes a 
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network driver layer, and a transport driver interface 
layer arranged to package applications files as packets 
capable of being routed over the open network and supply 
the packets to the network driver layer for transmission to 
the open network, and wherein said shim is a transport 
driver interface layer shim positioned between the 
applications program and the transport driver interface 
layer to intercept service requests by the applications 
program to the transport driver interface layer in order to 
cause the applications level authentication and encryption 
program to communicate with the server, generate said 
session key, and encrypt files sent by the applications 
program before the files are packaged by the transport 
driver interface layer. 

24. Computer software as claimed in claim 23, wherein said 
applications program is a peer-to-peer communications 
program, and wherein a peer application destination 
address, included in said intercepted requests for service, 
is diverted by the transport driver interface layer shim 
and supplied to the server during communications with the 
server, causing the service to establish a communications 
link with a peer application, mutually authenticate the 
peer application, and enable the peer application to 
reconstruct the session key in order to receive encrypted 
files sent by the peer-to-peer communications program over 
the open network. 
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25. Computer software as claimed in claim 23, further 
comprising a network driver layer shim positioned between 
the network driver layer and the transport driver interface 
layer and arranged to intercept files packaged by the 
transport driver interface layer and encrypt the files 
using a session key generated during communications with a 
lower layer of the server. 

26. Computer software for installation on a client 
computer of a multi-tier virtual private network, said 
network including a server and a plurality of client 
computers, the server and client computers each including 
means for transmitting data to and receiving data from an 
open network, 

wherein said computer software includes: 

applications level encryption and 
authentication software arranged to communicate 
with the server in order to: a. ) mutually 
authenticate the server and the client computer 
initiating communications with the server and b. ) 
generate a session key for use by the client 
computer initiating communications to encrypt 
files ; and 

at least one lower level set of 
communications drivers, 

wherein said lower level set of 
communications drivers includes a network driver 
layer, a transport driver interface layer 
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arranged to package applications files as packets 
capable of being routed over the open network and 
supply the packets to the network driver layer 
for transmission to the open network, and a 
network driver layer shim positioned between the 
transport driver interface layer and the network 
driver layer and arranged to intercept files 
packaged by the transport driver interface layer 
and encrypt the files using a session key 
generated during communications with a lower 
layer of the server. 

27. Computer software for installation on a client 
computer of a multi-tier virtual private network, said 
network including a server and a plurality of client 
computers, the server and client computers each including 
means for transmitting data to and receiving data from an 
open network, 

wherein said computer software includes: 
applications level encryption and authentication software 
arranged to communicate with the server in order to: a.) 
mutually authenticate the server and the client computer 
initiating communications with the server and b.) generate 
a session key for use by the client computer initiating 
communications to encrypt files; and 

further comprising means for securing peer-to-peer 
communications between applications on two of said client 
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computers, said peer-to-peer communications securing means 
comprising : 

means for intercepting a destination address 
during initialization of communications by a 
first of said two client computers; 

means for causing said authentication 
software to communicate with the server to carry- 
out functions a.) and b.); 

means for transmitting said destination 
address to said server; 

means for causing said server to carry-out 
functions a.) and b.) with respect to the second 
of said two client computers; 

means for enabling said second of said two 
client computers to recreate the session key; 

means for causing said authentication 
software to encrypt files to be sent to the 
destination address using the session key; 

means for transmitting the encrypted files 
directly to the destination address. 

28. Computer software as claimed in claim 27, wherein said 
means for intercepting the destination address comprises a 
shim positioned between the peer-to-peer applications 
program and a layer of a communications driver architecture 
of said first of the two client computers. 
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29. Computer software as claimed in claim 27, wherein said 
shim is positioned above a socket, the socket being 
positioned above a transport driver layer of said 
communications driver architecture. 

30. Computer software as claimed in claim 27, wherein said 
shim is positioned above a transport driver layer of said 
communications driver architecture. 

31. A method of carrying out communications over a multi- 
tier virtual private network, said network including a 
server and a plurality of client computers, the server and 
client computers each including means for transmitting data 
to and receiving data from an open network, comprising the 
steps of: 

intercepting function calls and requests for service 
sent by an applications program in one of said client 
computers to a lower level set of communications drivers; 

causing an applications level authentication and 
encryption program said one of said client computers to 
communicate with the server, generate said session key, and 
encrypt files sent by the applications program before 
transmittal over said open network. 

32. A method as claimed in claim 31, further comprising 
the step of intercepting files packaged by a transport 
driver interface layer to form packets and encrypting the 
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packets using a session key generated during communications 
with a lower layer of the server. 

33. A method as claimed in claim 31, further comprising 
the step of intercepting a destination address during 
initialization of communications between said one of said 
client computers and a second of said client computers on 
said virtual private network; 

causing said applications level 
authentication and encryption program to 
communicate with the server to carry out 
functions a.) and b.); 

transmitting said destination address to 
said server; 

causing said server to carry-out functions 
a.) and b.) with respect to the second of said 
two client computers; 

enabling said second of said two client 
computers to recreate the session key; 

causing said authentication software to 
encrypt files to be sent to the destination 
address using the session key; and 

transmitting the encrypted files directly to 
the destination address. 

34. A method as claimed in claim 33, wherein said step of 
intercepting the destination address is carried out by a 
shim positioned between a peer-to-peer applications program 
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and a layer of a communications driver architecture of said 
one of the two client computers. 
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ABSTRACT OF THE DISCLOSURE 



A virtual private network for communicating between a 
server and clients over an open network uses an 
applications level encryption and mutual authentication 
program and at least one shim positioned above either the 
socket, transport driver interface, or network interface 
layers of a client computer to intercept function calls, 
requests for service, or data packets in order to 
communicate with the server and authenticate the parties to 
a communication and enable the parties to the communication 
to establish a common session key. Where the parties to 
the communication are peer-to-peer applications, the 
intercepted function calls, requests for service, or data 
packets include the destination address of the peer 
application, which is supplied to the server so that the 
server can authenticate the peer and enable the peer to 
decrypt further direct peer-to-peer communications . 

NWB-B: \VPN.APP 



51 



1/7 

Client/Server VPN 




FIG. 1A 

(PRIOR ART) 



Peer-to-Peer Tunneling 




8 



FIG. IB 

(PRIOR ART) 



2/7 



36 




-26 




TDI Layer (transport protocols such as TCP/IP, UDP, 
Netbeui, IPX) 



31 



Datagrams, 
IP Packets, 
etc. 



•24 



Hardware Drivers (e.g. NDIS Drivers) 



32- 



Ethernet Packets, 
Analog Signals 



52. 



Hardware 



,25 



FIG. 2 

(PRIOR ART) 



3/7 



27. 




26 



TDI Layer (transport protocols such as TCP/IP, UDP, 
Netbeui, IPX) 


31^ 

\ 


Datagrams, 

IP Packets, 0A 
etc. / 24 

2 f— i 


Hardware Drivers 


(e.g. ND1S Drivers) 



32- 



Ethernet Packets, 
Analog Signals 



52. 



Hardware 



,25 



FIG. 3 



4/7 



27. 



App 1 



App 2 



App 3 



26 
20 




FIG. 4 



5/7 



27. 



36 



App 1 



App 2 App 3 



Peer-to- Peer 
Applications 



37- 



43 / 



/ 

' 29 



Authentication 
Client Software 
1 



-26 
20 



-54 



Peer-to-Peer 
Applications 



/ f i 

' 1 Socket Shim / V " 50 



42- 



\ 43 / 
/ 



\ / 30 



Socket 



'22 



TDI Shim 



53 



TDI Layer (transport protocols such as TCP/IP, UDP, 
Netbeui, IPX) 



31- 



21 



NDiS Shim 



Encrypted Packets 



55 



Hardware Drivers (e.g. NDIS Drivers) 



32- 



24 



Ethernet Packets, 
Analog Signals 



1Z. 



Hardware 



.25 



FIG. 5 



6/7 




7/7 

Applications Level Proxy 



100 

101 ■ 

102 
103 

104 
105 

106 
107 
108 
109 



Intercept Connection Requests, Function Calls, 
or Requests for Service Made by an 
Applications Program 



X 



Establish Communications Link to 
Authentication Server 



X 



Mutually Authenticate Client and Server 
and Generate Session Key Based on 
Shared Secret Keys 



X 



Encrypt Further Communications 



Peer-to-Peer 



Intercept Destination Address 



X 



Establish Communications with 
Authentication Server 



Perform Mutual Authentication and Encryption 



X 



Establish Communications Channel Between 
Authentication Server and Destination Client 



Enable Destination Client to Decrypt 
Files Sent by Peer Application 



X 



Encrypt Further Peer-to-Peer Communications 



FIG. 7 



ATTORNEY/ docx. <o: aUXit VOfxt; vrti 

Declaration for Patent Application and Appointment of Attorney 

As a below named inventor, I hereby declare that my residence, pott office address and citizenship are as itnT rd below next to my nam 
I believe that I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural nam 
are listed below) of the subject matter which is claimed and for which a patent is sought on the invention (Design, if applicable) -mrt-f- 

MULTI -ACCESS VIRTUAL PRIVATE NETWORK 

the specification of which (check one): 

□ is attached hereto, or S was filed on: August 26, 1997 

Application Number: 08/917,341 

and (if applicable) was amended on: 
I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as amended by asp 
amendments) referred to above. I acknowledge the duty to disclose information which is material to patentability as defined in Title 57. Cad 
of Federal Regulations, $1.56. I hereby claim foreign priority benefits under Title 35, United States Code %119 of any foreign a 
for patent or inventor's certificate listed below and have also identified below any foreign application for patent or im 

A filing datA he fit rg that rif th#» flnrili(*Bhnn nwt iuhir%U nnnn*i r -l..: 1 



at U.S. Application Number or PCT ] 



Prior Foreign Applications) 


Priority Claused 


Number 


Country 


Day/Month/Year Filed 


Yes 


Mm 























i HEREBY CLAIM THE BENEFIT UNDER TITLE 35 U.S. CODE 5 1 1 9(E) OF ANY U.S. PROVISIONAL APPLICATIONS LISTED BELOW. 



Application Number 



Day/Month/Year Filed 



" i_i Additional Provisional Appiicanon(s) Listed on Following Page(s) 

I hereby claim the benefit under Tale 35, United States Code, 1120 of any United States applications) or PCT international -re- 
designating The United States of America listed below and. insofar as the subject matter of each of the claims of this application is not > 
in that/those prior application(s) in the manner provided by the first paragraph of Title 35. United States Code, §JJ2, 1 acknowledge 
to disclose information which is material to patentability as defined in Tale 37, Code of Federal Regulations, %1.56 which became 
between the filing date of the prior application(s) and the national or PCT international filing date of this application: 



Application Number 


Filing Date 


Status - Patented, Pending or Abandoned 















I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information aad fac& 
are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so saute a? 
punishable by fine or imprisonment, or both, under section 1001 of title 18 of the United States Code and that such willful false statements m* 
jeopardize the validity of the application or any patent issued thereon. 

POWER OF ATTORNEY: I (We) hereby appoint as my (our) attorneys, with full powers of substitution and revocation, to pssaew. 
this application and transact all business in the Patent and Trademark Office connected therewith: J. Ernest Kenncy, Reg. No. 19.179; Biges 
Mar, Reg. No. 25,893; Richard E. Renter, Reg. No. 26,382; Charles R. Wolfe, Jr., Reg. No. 28,680; Thomas J. Moore, Reg. No. 28,974 
Bruce H. Troxell, Reg. No. 26,592; and 

I(we) authorize my(our) attorneys to accept and follow instructions from 

regarding any matter related to the preparation, examination, grant and maintenance of this application, any 



or divisional based thereon, and any patent resulting therefrom, until I(we) or my(our) assigns withdraw this aumorization in writing. 
Send correspondence to: BACON & THOMAS Telephone Calls to: BENJAMIN E. URCIA 

625 Slaters Lane - 4th Floor (703) 683-0500 

Alexandria. VA 22314 



Full Name of First or Sole Inventor 
James F. CHEN 


Citizenship 
U.S. 


Residence Address 

12648 Ikvilah Road, 
Potomac, Maryland 20854 


Post Ofrce Address is the same as Residence Ac 




D 'X I* "A. fa 





Continuation of Declaration for Patent Application and Appointment of Attorney 

Pase 2 



Prior Forsgn Applications) (35 USC S 1 1 9) 


Priority Claimed 


Number 


Country 


Day/Month/Year Filed 


Yes 


No 












1 









Prior Provisional Applications 35 U.S. Code §1 19(e) 


Application Number 


Day/Month/Year Filed 











Prior U.S. 


or PCT International Applications (35 U.S. Code §120) 




Application Number 


Filing Date 


Status • Patented. Peru 


ling or Abandoned 








1 





Full Name of Joint Inventor 
Jieh-Shan WANG 


Citizenship 

UJS. 


Residence Address 

10903 Silent Wood Place, 
N. Potomac, Maryland 20878 


Post Office Address is the same as Residence Address unless otherwise 








Full Name of Joint Inventor 

Christopher T. BROOK 


Citizenship 


Residence Address 

7308 Pomander Lane, 

Chevy Chase, Maryland 20815 


Post Office Address is the same as Resoence Address unless otherwise 


>T 2. c^-r (<*<<-} 






Full Name of Joint Inventor 
Franco GARVEY 


Citizenship 
U.S. 


Residence Address 

2908 S. Buchanan Street, 
Arlington, VA. 22206 


Post Office address is the same as Residence Address unless otherwise 
shown below 


Date / / 

X /o / z / jn- 


Signature /\ 




Full Name of Joint Inventor 


Citizenship 


Residence Address 


Post Office Address is the same as Residence Address unless otherwise 
shown below 


Date 


Signature 



Pat ap att. Dec (081396) 



□ See following pages for additional joint inventors/priority applications. 



